Application
This unit describes the skills and knowledge required to test captured transactional and non-transactional big data samples prior to using them in the organisation. It involves assembling or obtaining raw big data, processing that big data, and testing it in a way that enables it to be used more broadly within the organisation.
It applies to those who work in a broad range of industries using data analysis techniques in the management of their day-to-day work.
No licensing, legislative or certification requirements apply to this unit at the time of publication.
Elements and Performance Criteria
ELEMENT | PERFORMANCE CRITERIA |
1. Validate assembled or obtained big data sample | 1.1 Establish a sampling strategy for big data testing and identify a representative sample for big data testing 1.2 Assemble or obtain sample of raw big data according to legislative requirements and organisational policies and procedures 1.3 Validate big data sample from various sources to ensure that big data is correct |
2. Validate big data sample process and business logic | 2.1 Align datasets to relevant parts of the organisation 2.2 Implement data aggregation and segregation rules on a small set of sample data and datasets 2.3 Consult with required personnel to clarify and resolve identified anomalies 2.4 Conduct performance testing for data throughput, data processing and sub-component performance |
3. Validate output of captured big data sample and record results | 3.1 Design, formulate and select suitable test scenarios and test cases to validate output of big data sample 3.2 Implement selected test scenarios and test cases with big data sample using common testing tools and according to organisational procedures 3.3 Isolate sub-standard data and correct data acquisition paths as required 3.4 Generate and store results of validation activity and associated supporting evidence according to organisational policies and procedures, and legislative requirements |
4. Optimise big data sample results and documentation | 4.1 Perform data cleansing on big data sample following testing according to industry practices and organisational procedures 4.2 Collate validated output of testing, confirming absence of big data corruption in sample 4.3 Recommend configuration optimisation changes based on performance testing results 4.4 Communicate final sample results to required personnel |
Evidence of Performance
The candidate must demonstrate the ability to complete the tasks outlined in the elements, performance criteria and foundation skills of this unit, including evidence of the ability to:
test two different big data samples: one transactional and one non-transactional
conduct performance testing on two different big data samples: one transactional and one non-transactional.
Evidence of Knowledge
The candidate must be able to demonstrate knowledge to complete the tasks outlined in the elements, performance criteria and foundation skills of this unit, including knowledge of:
legislative requirements relating to testing big data sources, including data protection and privacy laws and regulations
industry protocols and procedures required to write queries and scripts for big data testing
organisational policies and procedures relating to testing big data sources, including:
assembling and obtaining raw big data
performing data cleansing following extract, transform and load (ETL) testing
isolating sub-standard data and correcting data acquisition paths
quality assuring output
testing transactional and non-transactional sources of big data
storing test results and associated support evidence
big data validation protocols, including:
big data testing methodologies
test scripting
features and formats of common big data sources, including:
batched
real time
interactive
protocols and techniques for:
performance testing big data throughput
processing and reporting issues.
Assessment Conditions
Skills must be assessed in a workplace or simulated environment where conditions are typical of a work environment that uses big data.
Access is required to:
information and telecommunications equipment required to test big data sources
big data sets to be tested
industry standards, organisational procedures, and legislative requirements required to demonstrate the performance evidence.
Assessors of this unit must satisfy the requirements for assessors in applicable vocational education and training legislation, frameworks and/or standards.
Foundation Skills
Skill | Description |
Learning | Modifies behaviour following exposure to new information |
Oral communication | Asks open and closed probing questions and actively listens to feedback during big data testing |
Numeracy | Interprets numerical data completes at times complex calculations and records numerical data |
Reading | Identifies and interprets information from relevant sources to complete work |
Writing | Uses clear, specific and industry-related terminology to represent test results |
Planning and organising | Efficiently and logically sequences the stages of big data testing |
Technology | Uses appropriate technology platforms and query languages and scripts to test big data |
Sectors
Data Literacy – Data Literacy